<html>
<head><meta charset="utf-8"><title>Coverage reports for compiler backend · t-compiler/help · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/index.html">t-compiler/help</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html">Coverage reports for compiler backend</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="234712529"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/234712529" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adrian Palacios <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#234712529">(Apr 15 2021 at 16:50)</a>:</h4>
<p>Hello all, we are working on a compiler backend for program verification and we would like to generate <a href="https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/source-based-code-coverage.html">source-based coverage reports</a> for this backend based on our testing suite.</p>
<p>The usage examples that I have seen rely on <code>cargo build</code> and <code>cargo test</code> to generate profiling data, but we use <code>x.py</code> for building and our own scripts for testing. Being a compiler backend, I think we cannot get rid of <code>x.py</code> for now. What do you recommend we do in our case?</p>



<a name="234712723"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/234712723" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#234712723">(Apr 15 2021 at 16:50)</a>:</h4>
<p><span class="user-mention" data-user-id="396031">@Adrian Palacios</span> RUSTFLAGS still works with x.py</p>



<a name="234819038"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/234819038" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adrian Palacios <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#234819038">(Apr 16 2021 at 09:03)</a>:</h4>
<p>Thanks <span class="user-mention" data-user-id="232545">@Joshua Nelson</span> !</p>
<p>I have tried to compile with <code>RUSTFLAGS="-Zinstrument-coverage" ./x.py build -i --stage 1 library/std</code> but I get the error:</p>
<div class="codehilite"><pre><span></span><code>error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn&#39;t exit successfully: `/local/home/adpaco/rmc/build/x86_64-unknown-linux-gnu/stage0/bin/rustc - --crate-name ___ --print=file-names -Zinstrument-coverage -Cdebuginfo=2 -Wrust_2018_idioms -Wunused_lifetimes --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit code: 1)
  --- stderr
  error: the option `Z` is only accepted on the nightly compiler

failed to run: /local/home/adpaco/rmc/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /local/home/adpaco/rmc/src/bootstrap/Cargo.toml
Build completed unsuccessfully in 0:00:00
</code></pre></div>
<p>I thought we were already building a nightly version of the compiler. I tried some options in the <code>config.toml</code> file (e.g., <code>channel = nightly</code>) but none of them have worked. What do I need for this option to be accepted?</p>



<a name="234820781"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/234820781" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#234820781">(Apr 16 2021 at 09:16)</a>:</h4>
<p>The bootstrap compiler is a beta compiler. It should accept nightly options through the bootstrap trick though.</p>



<a name="234840176"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/234840176" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#234840176">(Apr 16 2021 at 11:57)</a>:</h4>
<p>I don't think we always pass the bootstrap trick, as we try to limit exposure of e.g. bootstrap tools. You might be able to get it working by passing RUSTC_BOOTSTRAP=1 yourself, but I would recommend switching to a nightly compiler via build.rustc and build.cargo in config.toml</p>



<a name="234846452"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/234846452" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#234846452">(Apr 16 2021 at 12:46)</a>:</h4>
<p><span class="user-mention" data-user-id="396031">@Adrian Palacios</span> you want to only set the option when  building the compiler itself - try <code>RUSTFLAGS_BOOTSTRAP</code></p>



<a name="236711950"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/236711950" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adrian Palacios <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#236711950">(Apr 29 2021 at 17:20)</a>:</h4>
<p>Thank you all for your help! Using the <code>RUSTFLAGS_BOOTSTRAP</code> and switching to nightly versions of rustc/cargo seems to have worked.</p>
<p>That said, I have not been able to run it so that instrumented data (i.e., <code>.profraw</code> files) is generated for our tests. Running <code>./x.py test</code> generates some of these files, then fails to complete since we do not support rust tests. Nothing is generated if I use the standalone <code>rustc</code> that we build. What should we do in this case?</p>



<a name="236712156"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/236712156" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#236712156">(Apr 29 2021 at 17:21)</a>:</h4>
<p>I don't understand the question - what do you mean you don't support rust tests? How do you expect to get coverage without running tests?</p>



<a name="236714724"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/236714724" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adrian Palacios <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#236714724">(Apr 29 2021 at 17:38)</a>:</h4>
<p>We do not use the compiletest tests from <code>/src/test</code> - our tests are located in other folders and we have different scripts for running them.<br>
I suppose one way is to edit the <code>src/bootstrap/test.rs</code> file to run our tests instead, but I would like to know if there are alternative ways.</p>



<a name="236714855"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/236714855" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#236714855">(Apr 29 2021 at 17:38)</a>:</h4>
<p>you could see how compiletest does it and then do it the same way in your own test suite</p>



<a name="248579601"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/248579601" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> jmanchuck <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#248579601">(Aug 06 2021 at 07:26)</a>:</h4>
<p>Hi <span class="user-mention" data-user-id="396031">@Adrian Palacios</span>, did you manage to successfully get instrumented data in the end? I'd like to test lines/% coverage of the Rust compiler from its tests - seems like you were doing the same thing</p>



<a name="248602084"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Coverage%20reports%20for%20compiler%20backend/near/248602084" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adrian Palacios <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Coverage.20reports.20for.20compiler.20backend.html#248602084">(Aug 06 2021 at 12:05)</a>:</h4>
<p>Hey <span class="user-mention" data-user-id="424037">@jmanchuck</span> we have been busy porting our testing suite to compiletest and this is the next item on my to-do list. I can post updates here if that helps, but next week I am on vacation so it will have to wait until then.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>